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ABSTRACT 


This  thesis  explores  the  theoretical  and  practical  prob- 
lems encountered  in  using  Multiple  Attribute  Utility  Theory 
to  assign  capability  measures  to  military  weapon  systems. 

Two  experiments  were  conducted,  the  first  employed  the 
classical  form  of  data  representation  and  the  second  utilized 
piecewise  linear  approximations  as  a practical  alternative. 

It  was  concluded  that  Multiple  Attribute  Utility  Theory 
appears  to  be  a significant  step  forward  in  the  search  for 
a valid  measure  of  arms  transfers. 
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I.  INTRODUCTION 

Centuries  ago,  the  transfer  of  arms  from  one  nation  to 
another  was  a relatively  simple  process  with  consequences 
the  average  man  could  easily  understand.  A bow  and  arrow 
involved  one  man,  had  an  estimable  reloading  time  and  could 
kill  up  to  a measureable  distance.  Today,  however,  modern 
technology  has  produced  a class  of  weapons  whose  complexity 
and  sophistication  defy  the  simple,  subjective  calculations 
that  faced  the  merchants  and  analysts  of  primitive  weapons. 

One  needs  something  other  than  intuition  or  a "gut 
feeling"  to  reliably  assess  the  military,  political,  and 
economic  implications  of  selling,  say,  twenty  F-15  "Eagles" 
to  a nation  in  the  Middle  East.  Yet,  it  is  in  this  arena 
of  reliably  assessing  military  capabilities  among  third 
world  nations  that  some  of  the  most  pressing  methodological 
problems  exist.  Dollar  valuations,  inventory  comparison  and 
military  utility  have  all  been  examined  and  to  date,  have 
not  provided  a satisfactory  indicator  of  military  capability. 

As  an  alternative.  Multiple  Attribute  Utility  Theory 
(MAUT)  appears  to  provide  a more  consistent  estimator.  MAUT 
is  an  analytic  device  with  roots  in  the  field  of  economics. 

It  combines  a class  of  psychological  measurement  models  and 
scaling  techniques  to  distill  multi-dimensional  alternatives 
into  a single,  ratio  index. 


This  thesis  will  examine  the  theoretical  and  practical 
problems  encountered  in  using  MAUT  to  assign  capability 
measures  to  weapon  systems  which  take  into  account  the  vari- 
ables that  are  relevant  in  a third  world  context.  Research 
has  been  confined  to  the  analysis  of  air-to-air  weapon 
systems  but  the  analogy  to  other  platforms  is  readily  made. 
All  data  and  results  are  presented  so  that  further  research 
may  begin  from  this  point. 

Chapter  II  reviews  some  of  the  dominant  methodologies 
that  have  appeared  to  date.  It  also  discusses  Multiple 
Attribute  Utility  Theory  with  its  pertinent  forms  and 
assumptions.  Additionally,  it  introduces  the  factors  con- 
sidered to  be  descriptive  cf  fighter  aircraft.  Chapter  III 
presents  a small  sample  survey  in  which  the  necessary  data 
for  applying  MAUT  to  fighter  aircraft  is  presented.  The 
data  are  used  to  evaluate  the  F4J  "Phantom"  with  Israeli 
and  Egyptian  pilots.  The  respondents  for  this  survey  were 
experienced  aviators  who  possessed  a knowledge  of  utility 
curves  and  their  uses.  Chapter  IV  examines  a possible 
alternative  to  the  classical  form  of  data  collection . 

Instead  of  the  required  utility  curves,  piecewise  linear 
approximations  are  substituted.  These  approximations  con- 
tain three  critical  points  and  were  derived  from  the 
responses  of  200  members  of  the  Red  River  Valley  Fighter 
Pilots  Association.  It  is  hoped  this  technique  will  pro- 
vide the  policy  maker  with  a relatively  simple  procedure 
for  collecting  data  and  simultaneously  broadening  his 


data  base.  Chapter  V summarizes  the  major  conclusions  and 
observations  noted  during  the  course  of  this  research. 
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II.  METHODOLOGIES  FOR  ASSESSING  ARMS  TRANSFERS 


A.  REVIEW  OF  CURRENT  METHODOLOGIES 

This  section  will  summarily  review  the  dominant  method- 
ologies for  measuring  arms  transfers.  These  are  the  dollar 
value,  numerical/inventory  comparisons  and  military  capabil- 
ity approaches.  It  is  hoped  that  this  brief  discussion  will 
generate  an  appreciation  for  the  difficulties  encountered  in 
reliably  assessing  arms  transfers. 

1 . Dollar  Value  Method 

In  1969,  the  Stockholm  International  Peace  Research 
Institute  (SIPRI) [1]  began  a formal  effort  to  monitor  arms 
flows  between  the  rich  industrial  powers  and  the  rest  of 
the  world.  SIPRI  employed  the  dollar-value  technique  and 
collected  their  data  "from  a wide  variety  of  sources. 
Basically,  analysts  employing  the  dollar-value  technique  use 
price  to  quantify  the  volume  and  direction  of  arms  flow.  At 
first  glance,  this  may  seem  to  be  a reasonable  measure  of 
capability  since  price  information  is  understood  easily,  is 
measured  on  a ratio  scale  (thus  enjoying  all  the  properites 
of  the  real  number  line)  and  is  available  in  most  instances. 
A closer  examination,  however,  reveals  four  major  weaknesses 
of  this  method. 

a.  Inflation  and  fluctuating  exchange  rates  often 
precipitate  a situation  where  the  increase  in  arms 

'''Stockholm  International  Peace  Research  Institute,  The 
Arms  Trade  with  the  Third  World,  p.  VI,  Almquist  and 
Wilksells  3oktryckeri  AB,  Uppsala,  1971. 
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expenditures  corresponds  to  a decrease  in  the  number  of 
weapons  actually  transferred.  Analysts  have  attempted  to 
alleviate  this  problem  by  adjusting  yearly  figures  to  reflect 
some  arbitrary  exchange  rate.  Unfortunately,  these  fluctu- 
ations usually  are  so  rapid  they  cannot  be  compensated  for 
and  the  weapon's  true  value  at  the  time  of  the  transaction 
is  unclear. 

b.  Much  uncertainty  exists  regarding  cost  data  for 

foreign  arms  transfers  — particularly  in  communist  countries. 

When  this  information  is  lacking  or  insufficient,  a tendency 

exists  to  evaluate  these  figures  in  terms  of  Western  produc- 
2 

tron  costs.  As  Sivard  states,  " ...  although  statistical 
work  on  such  parity  rates  is  underway,  under  international 
sponsorship,  the  availability  of  purchasing  power  parities 
for  a large  selection  of  countries  is  some  distance  in  the 
future."^  Hence,  an  acknowledged  doubt  exists  concerning 
foreign  cost  data  and  its  reliability. 

c.  In  many  instances,  arms  are  sold  from  one  nation 

to  another  for  as  little  as  one-tenth  to  one-hundredth  of 

4 

their  initial  cost.  Often,  the  reasons  for  such  favorable 
terms  are : 


2 

The  International  Transfer  of  Conventional  Arms , p . 1 , 
Washington:  Arms  Control  and  Disarmament  Agency,  1973. 

"^Sivard,  Ruth  L.  , World  Military  and  Social  Expenditures 
1974 , p.  30,  WMSE  Publications,  1974. 

4 

Mihalka,  Michael,  Understanding  Arms  Accumulation:  The 
Middle  East  As  An  Example,  p.  14,  University  of  Michigan 
(MIMEO),  1973. 
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(1)  the  supplier  country  may  be  depleting 


excess  stock; 

(2)  the  supplier  country  may  no  longer  have 
any  use  for  that  particular  equipment;  or 

(3)  the  transfer  of  arms  may  be  a rider  to  a 
much  larger  deal. 

Clearly,  the  real  cost  of  such  transactions  has 
been  obscured  by  the  extremely  favorable  terms  of  sale. 

d.  Finally,  price  is  an  unreliable  indicator  of 
the  qualitative  differences  in  arms.  Consider  the  F-14  and 
the  C5A.  They  are  both  examples  of  costly  military  systems 
yet  a marked  difference  exists  in  their  military  offensive 
power.  Gross  differences  such  as  these  need  to  be  discerned 
if  one  is  to  properly  assess  the  true  military  worth  of  arms 
transfers . 

2 . Numverical/Inventory  Approach 

With  the  numerical/inventory  approach,  countries' 
capabilities  are  compared  on  the  basis  of  their  respective 
inventories,  say,  of  MIGs  and  Phantoms.  In  some  respects, 
this  approach  is  more  reliable  than  the  dollar-value  method 
since  major  weapons  are  difficult  to  hide  and  a country's 
supply  of  these  items  are  usually  public  knowledge.  However, 
correlating  inventories  with  capability  is  difficult  because 
of  the  qualitative  differences  among  the  various  weapons 
systems.  Just  how  much  better  is  an  F4E  with  a pilot  from 
Country  A than  a MIG-19  with  Country  B's  pilot?  Certainly, 
lessons  learned  from  Vietnam  and  the  two  Middle  East  wars 
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should  confirm  the  notion  that  inventory  balance  sheets  are 
not  a reliable  indicator  of  military  capability. 


3 . Military  Capability  Models 

In  an  effort  to  overcome  the  shortcomings  of  the 
dollar-value  and  numerical/inventory  methodologies,  analysts 
have  employed  various  indexing  techniques  for  juxtaposing 
weapons  systems  along  some  capability  scale  [2] . 

One  of  the  most  intuitively  pleasing  approaches  to 
scaling  military  capability  is  the  factor  analysis  technique. 
This  process  synthesizes  a collection  of  interval  data  into 
a set  of  summary  dimensions  and  shows  the  degree  to  which 
discrete  variables  are  associated  with  each  summary  dimension. 
Mihalka  [3]  and  Snider  [4],  proponents  of  this  approach, 
employ  similar  hypotheses  whereby  aircraft  are  categorized 
into  two  summary  dimensions.  Milhalka  considers  fighter 
aircraft  to  fall  somewhere  along  the  two  different  combat 
missions  of  "attack"  and  "defense"  while  Snider  feels  they 
should  be  separated  into  "interception/air  superiority"  and 
"tactical  support  ground  attack."  After  making  these  a 
priori  decisions  about  aerial  combat  dimension,  Mihalka  and 
Snider  separately  factor  analyzed  various  data  sets  and  were 
able  to  reduce  each  aircraft  to  a single,  interval  score 
(an  example  of  Snider's  aircraft  scores  is  provided  in 
Figure  1).  These  interval  scores,  however,  are  inherently 
unstable.  This  is  because  there  is  no  natural  or  fixed 
origin  with  interval  measures.  For  example,  consider  the 
Farenheit  scale  used  to  measure  temperature.  The  "zero" 
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for  this  system  was  arbitrarily  chosen  to  fall  32  (Faren- 
heit)  degrees  below  the  freezing  point  of  water.  It  could 
have  easily  been  31  or  33  Farenheit  degrees  below  the 
freezing  point  of  helium.  Since  there  is  never  a lack  of 
temperature,  there  is  virtually  an  infinite  number  of  zero 
points  for  this  system  or  any  system  using  interval  measure- 
ment. By  contrast,  techniques  that  achieve  ratio  measure- 
ment have  an  inviolate  zero  point,  identical  in  every 
respect  to  the  zero  of  the  real  number  line.  Thus,  analysts 
evaluating  the  various  methodologies  measuring  arms  flows 
must  be  critical  of  the  theory  and  the  level  of  measurement 
that  the  particular  technique  achieves.  As  stated  earlier, 
factor  analysis  does  not  produce  ratio  data  and  perhaps  the 
problems  generated  by  such  approaches  are  best  illustrated 
by  the  following  example. 

"Consider,  first,  the  process  leading  to  the  aircraft 
inventory  capability  scores.  The  first  step  of  this 
calculation  involves  adjusting  the  derived  weapons 
capability  scores  so  that  there  are  no  negative  or 
zero  values.  Mihalka  accomplishes  this  by  adding  0.1 
(selected  arbitrarily)  to  the  absolute  value  of  the 
lowest  factor  score  and  adding  the  resulting  sum  to 
each  aircraft  score.  The  effect  is  to  move  each  system 
in  a positive  direction  along  the  interval  scale  by  the 
amount.  Recall  that  this  is  permissible  with  interval 
measurement  since  the  information  is  preserved  by  a 
linear  transformation.  Multiplying  these  adjusted 
values  by  varying  inventories  to  obtain  composite 
country  scores  is  tenuous,  however  ...  An  example  will 
illustrate  this.  Suppose  the  derived  factor  score  for 
aircraft  A is  2.0  and  for  aircraft  B,  1.0,  along  the 
offensive  dimension.  If  a country  had  an  inventory  of 
25  As,  the  country  capability  score  would  be  25  x 2.0  = 
50.0.  Similarly,  if  a second  country  had  50  Bs , its 
capability  score  would  also  be  50.0.  Clearly,  this 
would  be  a situation  of  parity.  Now  consider  the  trans- 
formation of  the  individual  factor  scores  by  an  arbitrary 
value  of  0.5,  i.e.,  aircraft  A = 2.5  and  aircraft  B = 1.5. 
Multiplying  these  adjusted  capability  scores  by  the  same 
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country  inventories  yields  a capability  score  of  67.5 
for  the  first  (25  x 2.5  = 67.5),  and  75.0  for  the 
second  (50  x*  1.5  = 75).  A situation  of  equality  has 
suddenly  become  an  advantage  for  the  second  country 
with  any  change  in  the  number  or  type  of  weapons."* 

In  summary,  the  problem  of  reliably  measuring  arms 
transfers  remains  unsolved  [5].  Thus,  new  methodologies 
for  solving  this  problem  are  constantly  being  evaluated  and 
one  such  methodology,  Multiple  Attribute  Utility  Theory,  is 
discussed  and  examined  in  the  following  chapters. 


B.  MULTIPLE  ATTRIBUTE  UTILITY  THEORY 
1.  Rationale 

Theoretically,  Multiple  Attribute  Utility  Theory 
combines  a class  of  psychological  measurement  models  and 
scaling  techniques  that  can  be  applied  to  the  decision  mak- 
ing process  when  an  assessment  of  multi-facet  alternatives 
is  necessary.  Practically,  MAUT  provides  the  analyst  with 
a methodology  for  measuring  arms  transfers  on  a ratio  scale 
yet  avoids  the  difficulties  encountered  by  the  dollar  value, 
numerical/inventory,  and  military  capability  techniques. 

MAUT  does  this  by  first,  decomposing  weapon  systems  into 
their  basic  elements,  second  by  ascertaining  the  utility 
curve  and  factor  weight  for  each  element,  and  third,  by 
constructing  a model  to  evaluate  weapon  systems  of  the  same 
class.  Once  this  process  is  complete,  MAUT  enables  the 


Legrow,  Allan  W. , Measuring  Aircraft  Capability  for 
Military  and  Political  Analysis,  p.  36,  Masters  Thesis, 
Naval  Postgraduate  School,  Monterey,  CA,  March  1976. 
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analyst  to  "score"  a particular  weapon  system,  say  fighter 
aircraft,  on  the  basis  of  its  platform  design,  weapon  capa- 
bilities, pilot  experience  and  the  technical  level  of  the 
consumer  nation.  For  example,  suppose  Country  A has  a 
fighter  aircraft  with  a score  of  10  and  Country  B has  a 
fighter  aircraft  with  a score  of  5.  Then  Country  A has  the 
superior  fighter  and  Country  B would  need  two  of  their  air- 
craft  for  every  one  of  Country  A's  to  achieve  parity.  It 
should  be  noted,  however,  that  these  indices  are  only  mean- 
ingful to  aircraft  of  identical  classes.  That  is,  no  con- 
clusions can  be  drawn  from  an  air-to-ground  system  with  a 
score  of  7 and  an  air-to-air  system  with  a score  of  14, 
since  the  elements  underlying  each  evaluative  model  are 
dif rerent . 

2 . Theory/Application 

There  are  two  basic  forms  [6]  to  the  Multiple 
Attribute  Utility  Function  and  they  are: 
a.  The  Multiplicative  Form 

N 

1 + K • U (X)  = tt  [1  + K • K . *U . (X . ) ] (1) 

C—  1 111 

where  U(X)  = multi-attributed  utility  function  (In  fighter 

aircraft,  U(X)  would  designate  the  entire  system.) 

U(X^)  = the  utility  function  for  attribute  i 
X^  = a specific  attribute 
K = constant;  -1  <.  k < 00 

= subjective  weight  for  X^;  0 <.  <.  1 (If  attri- 

bute i were  dash  speed,  then  represents 

J 
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dash  speed's  weight  or  importance  to  the 
functioning  of  the  entire  system.) 

b.  The  Additive  Form 

N 

U(X)  = Z K . *U.  (X.  ) (2) 

i=l  1 1 1 

where  U(X)  = multi-attributed  utility  function 
U(X^)  = utility  function  for  attribute  i 

X^  = a specific  attribute 

= subjective  constant  for  U(X^);  0 <.  £ 1; 

N 

Z K.  = 1 
i=l  1 

Each  form  requires  the  same  data  for  input,  that 
is,  a utility  curve  and  a factor  weighting  for  each  factor. 
The  different  forms  reflect  the  combinatorial  relationship 
the  factors  bear  towards  one  another.  Before  discussing 
the  two  forms  and  their  attendant  properties,  a discussion 
of  the  data  and  how  they  were  collected  will  appear  first. 
The  discussion  begins  with  the  general  concepts  encountered 
in  utility  theory  and  concludes  with  the  factor  weighting  of 
each  factor. 

c.  Utility  Curves 

The  utility  curve  is  a concept  generally  encoun- 
tered in  the  field  of  economics.  It  is  a graphic  tool  that 
displays  the  user's  preference  or  utility  between  alterna- 
tives. It  entails  two  basic  assumptions,  (1)  the  person 
who  draws  such  a curve  will  always  act  in  a rational 
manner;  and  (2)  this  person  is  aware  of  his  alternatives.7 

7 Henderson  and  Quandt,  Microeconomic  Theory,  p.  6, 
McGraw-Hill,  2nd  Edition,  1971. 
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These  assumptions  have  precipitated  much  con- 
troversy in  the  professional  literature.  It  is  often 
argued  that  they  are  naive  and  unrealistic.  The  theory, 
however,  does  not  imply  these  assumptions  always  hold,  it 
merely  states  that  if_  they  hold,  the  derived  utility  func- 
tions are  valid.  It  is  up  to  the  person  who  employs  such 
techniques  to  determine  if  the  assumptions  of  rationality 
apply  in  a particular  analytic  situation.  For  the  purposes 
of  this  research,  it  is  assumed  these  assumptions  are 
satisfied  where  needed. 

Mapping  utility  curves  in  Multiple  Attribute 
Utility  Theory  requires  a series  of  difficult  decisions. 
One,  in  particular,  leads  to  the  derivation  of  ratio  meas- 
urement, a level  of  measurement  previously  identified  as 
crucial  in  the  measurement  of  arms  transfers.  To  illus- 
trate, a sample  curve  (Figure  2)  depicting  the  dash  speed 
of  a fighter  aircraft  is  presented  and  discussed. 


The  vertical  axis  represents  the  judge's  utility  scale  and 
is  annotated  from  0 to  1.  Zero  implies  no  utility  or  value 
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and  1.0  implies  the  maximum  utility  or  value.  The  horizon- 
tal axis  represents  the  factor  being  evaluated.  In  this 
case,  the  factor  is  dash  speed  of  a fighter  aircraft  (air- 
to-air)  and  the  speeds  range  from  MACH  .8  to  MACH  3.2.  An 
interpretation  of  this  curve  would  reveal  its  author  felt 
that  if  the  dash  speed  of  an  aircraft  was  less  than  MACH  .8 
it  had  no  utility  as  a fighter  aircraft.  In  other  words,  a 
dash  speed  less  than  MACH  .8  would  be  unacceptable  in  the 
fighter  community.  Similarly,  a dash  speed  of  MACH  2.4 
represents  the  optimum  speed  for  this  factor.  Any  further 
increases  would  be  of  marginal  value.  The  shape  of  the 
curve  also  conveys  much  information.  The  sharp  rise  from 
MACH  .8  to  MACH  1.6  indicates  large  capability  shifts  per 
unit  increase  in  speed.  If  the  vertical  scale  was  changed 
to  dollars,  the  analogy  would  infer  a greater  return  on  the 
dollar  per  unit  increase  in  speed  than  anywhere  else  on 
the  graph.  Similarly,  the  slow  rise  from  MACH  1.6  to 
MACH  2.4  (the  optimum)  indicates  small  capability  shifts  per 
unit  increase  of  speed. 

Deriving  such  curves  is  a stringent  process. 

The  steps  of  the  process  are  summarized  as  follows: 

(1)  Determine  an  upper  and  a lower  limit  for 
each  attribute.  The  lower  limit  should  represent  the  point 

g 

of  class  distinction  and  the  upper  limit,  a value  such 

g 

That  is,  if  the  attribute  were  to  assume  a smaller  value 
it  would  fall  into  a different  class.  For  example,  if  a 
race  car  has  a lower  limit  of  120  mph,  this  would  mean  any 
car  which  could  not  travel  at  least  120  mph  would  not  be  in 
the  class  of  race  cars,  it  would  be  in  the  class  of  passenger 
cars,  etc. 
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that  any  further  increases  would  only  result  in  marginal 
returns.  The  lower  limit  is  crucial  since  it  established 
the  property  of  ratio  measurement.  The  assumption  required 
is  that  the  lower  limit,  by  definition,  represents  a "natural" 
fixed  origin.  That  is,  as  far  as  the  judge  is  concerned, 
this  point  represents  the  absolute  zero  point  for  the  attri- 
bute in  question.  Thus,  if  the  reader  considers  this  a 
judicious  assumption,  ratio  measurement  has  been  achieved. 

(2)  The  body  of  the  curve  is  ascertained  through 
a series  of  probabilistic  decisions.  For  example,  if  the 
analyst  was  attempting  to  establish  the  value  associated 
with  the  50th  utile  (i.e. , the  dash  speed  that  corresponds 
to  . 5 on  the  vertical  axis)  the  analyst  would  present  the 
judge  with  two  theoretical  alternatives,  say,  A and  B.  The 
contents  of  alternative  A are  defined  by  the  utile  con- 
sidered and  usually  represent  the  extreme  cases.  Alternative 
B is  defined  by  the  judge  (i.e.,  an  expert  in  the  particu- 
lar field  of  interest)  and  the  decision  for  this  alternative, 
when  contrasted  with  alternative  A,  establishes  the  desired 
utile.  Using  the  values  of  this  example,  alternative  A 
represents  the  situation  where,  if  chosen,  there  is  a 50% 
chance  the  factor  would  be  designed  into  an  aircraft  co 
reflect  the  optimum  (MACH  2.4)  and  a 50%  chance  the  factor 
would  be  designed  into  an  aircraft  at  its  minimum  value 
(MACH  .8).  Alternative  B,  on  the  other  hand,  would 
guarantee  the  factor  be  designed  into  the  aircraft  at  a 
value  of  "X".  This  value  of  "X”  is  determined  by  the  judge. 
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It  should  reflect  a value  that  renders  the  judge  indiffer- 
ent between  alternative  A and  B. 


Pictorially,  the  alternatives  would  appear 

-50%  chance  factor  will  have  the  maximum 
value 


50%  chance  factor  will  have  the  minimum 
value 


R ^ 100%  guarantee  factor  will  have  some 

^ value  "X" 

The  judge  must  decide  what  value,  if  substituted  for  "X" 

would  make  him/her  indifferent  between  alternative  A and 

B.  Whatever  value  is  determined  for  "X"  is  the  50th  utile. 

In  other  words,  the  analyst  is  making  the  judge  establish 

the  point  where  the  risk  involved  in  alternative  A seems 

equally  attractive  as  the  "sure  thing"  of  alternative  B. 

If  the  analyst  were  attempting  to  establish  the  60th  utile, 

the  alternatives  would  appear  as 

•^60%  chance  factor  will  have  the  maximum 
value 


*40%  chance  factor  will  have  the  minimum 
value 


B =« >100%  guarantee  factor  will  have  some 

value  "X" 

(If  it  was  the  40th  utile,  the  probabilities  in  alternative 
A would  be  reversed.) 
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The  judge  is  informed  that  "indifference" 
between  alternatives  A and  B means  he/she  would  accept  the 
flip  of  a coin  to  determine  whether  the  consequences  of 
alternative  A or  alternative  B are  to  establish  value  of  the 
said  factor.  The  judge  is  further  informed  that  the  ramifi- 
cation of  his  choice  will  be  reflected  into  future  combat 
aircraft. 

d.  Subjective  Weights  (K^ ) 

Having  the  judge  subjectively  evaluate  each 
attribute's  contribution  to  the  overall  process  entails  the 
second  major  assumption  for  ratio  measurement.  In  short, 
judgmental  measurement  theory  hypothesizes  that  a definite 
regularity  exists  in  value  judgments.  Thus,  it  is  argued 
that  the  judge's  decision  reflect  enough  precision  to 
achieve  ratio  measurement.  Torgerson  [7] , however,  states 
that  some  judgmental  measuring  techniques  such  as  the  subjec- 
tive estimate  and  constant  sum  methods  actually  produce 
ratio  measurements.  Once  the  judge  has  evaluated  each  fac- 
tor's weight  and  utility  curve,  the  analyst  then  faces  the 
task  of  combining  these  judgments  into  a final  utility  score 
for  the  entity  being  assessed  (e.g.,  fighter  aircraft). 

This  is  done  by  determining  which,  if  any,  of  the  following 

properties  have  been  satisfied.  They  are,  as  defined  by 

_ . 9 

Giaque  : 

9 

Giaque,  William  C.,  Prevention  and  Treatment  of  Strep- 
tococcal Sore  Throat  and  Rheumatic  Fever  - A Decision 
Theoretic  Approach,  p.  10-18,  Ph.D.  Thesis,  Harvard  University, 
November  1972. 
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(1)  Utility  Independence.  Assume  consequence 

x = (x1,  xN)  with  utility  u(X) . if  Xj  = (x1#  xi_1> 

Xi+1'  XN'  ' Xj_  utility  independent  of  X-r-  if  the 

decision  maker's  relative  preference  for  X^  with  Xj  held 
fixed  are  the  same  regardless  of  the  chosen  value  of  Xj-. 

(2)  Pairwise  Preferential  Independence.  This 

property  exists  if  the  choice  between  two  consequences 
(X1  X 2 / X ^ , •••/  ) and  ( X^  , Xg , •••  X^j ) does  not 

depend  on  the  values  of  X^,  XN,  for  all  pairs  of 

attributes . 

(3)  Pairwise  Marginality.  Pairwise  marginality 

exists  if  the  lottery  (choice)  (X^,  X^),  (X?,  X^)  is 
indifferent  to  (X^,  X*)  , (X*,  X^),  where  lottery  A , B is  a 

choice  situation  with  the  probabilities  of  consequence  A 
and  B both  one-half. 

If  the  first  two  properties  hold,  the  multipli- 
cative form  is  used.  If  all  three  properties  hold,  the 
additive  form  is  used.  If  none  of  the  properties  can  be 
established,  the  technique  is  invalid.  However,  Keeney 
suggests  the  additive  form  may  be  applied  to  the  data  as 
this  form  provides  a good  approximation  of  the  multiple 
attribute  utility  function. ^ 


Keeney,  R.  L. , Multidimensional  Utility  Functions: 

Theory,  Assessment  and  Application,  p~!  2 3-2  5,  MIT, 

October  1969. 
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3 . Factors  Describing  Air-to-Air  Weapon  Systems 

It  is  the  intention  of  this  thesis  to  apply  the 
principles  of  Multiple  Attribute  Utility  Theory  to  air-to- 
air  fighter  aircraft.  As  stated  in  Section  1 of  this 
chapter,  the  study  of  any  system  requires  the  analyst  to 
decompose  the  system  into  its  basic  elements  or  factors. 
The  factors  used  in  this  study  reflect  the  technical 
research  done  by  Legrow  in  which  the  descriptive  factors 
for  fighter  aircraft  were  as  follows^: 


PLATFORM  CRITERIA 

WEAPON  CRITERIA 

MISCELLANEOUS  CRITERIA 

Max  Speed  (energy) 

Range 

Pilot  Proficiency 

Acceleration  (T/W) 

Missile  Speed 

Technological  Level 
of  the  country 

Manuever ability 
(W/S) 

Missile  Envelope 

Endurance 

(combat  radius) 

Number  of  Gun 
Barrels 

As  shown  above,  the  factors  contributing  to  capa- 
bility are  broken  into  three  major  dimensions;  platform 
criteria,  weapon  criteria,  and  miscellaneous  criteria.  In 
the  following  chapters  (III  and  IV),  these  factors,  exclud- 
ing the  technological  level  of  the  country,  are  presented  to 
experienced  aviators  in  order  to  ascertain  the  appropriate 
utility  curve  and  factor  weighting  for  each  factor.  All 
data  and  results  are  presented  so  that  further  research  may 
begin  from  this  point. 


■'■■'’Legrow,  o£.  cit.  , p.  122. 
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III.  SMALL  SAMPLE  SURVEY 


This  chapter  applies  the  principles  and  concepts  of 
Multiple  Attribute  Utility  Theory  to  the  problem  of  relia- 
bly assessing  arms  transfers.  This  is  done  by  conducting 

I 

a small  sample  survey  in  which  experienced  aviators  were 
instructed  to  determine  the  utility  curves  and  weights  for 
the  nine  descriptive  factors  of  fighter  aircraft,  described 
at  the  end'  of  the  preceding  chapter.  Section  A discusses 
the  main  source  of  data  collection  — a two  part  question- 
naire. Section  B describes  a step-by-step  application  of 
the  acquired  data  to  the  F4J  "Phantom"  using  Israeli  and 
Egyptian  pilots.  Section  C discusses  the  problems  encount- 
ered during  the  course  of  this  experiment  in  a "lessons 
learned"  format. 

A.  DATA  COLLECTION 

A questionnaire  (see  Appendix  A)  was  the  primary  source 
of  data  collection  for  this  experiment.  It  consists  of  two 
parts,  the  first  ascertaining  the  factor  utility  curves  and 
the  second,  the  factor  weights.  Since  the  respondents 
(students  at  the  Naval  Postgraduate  School)  were  familiar 
with  utility  curves  and  their  uses,  the  directions  for  part 
one  are  rather  straight  forward  and  focus  primarily  on 
graph  familiarization.  There  is  a short  note,  however,  that 
instructs  the  respondent  to  ignore  the  relationship  the 
factors  bear  towards  one  another  as  they  draw  each  utility 
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curve.  That  is,  if  the  respondent  was  considering  the  factor 
of  manueverability , the  potential  trade-offs  between  manuever- 
ability  and,  say,  dash  speed  should  be  ignored.  This 
instruction  was  intended  to  establish  the  property  of  utility 
independence  for  each  factor.  This  property  is  the  only 
one  of  the  three  that  either  exists  or  does  not  exist.  The 
other  two  properties  (pairwise  preferential  independence  and 
pairwise  marginality)  involve  lotteries  (choices)  and  it  was 
felt  these  properties  were  best  discussed  orally  with  the 
respondents . 

Part  two  employs  a "pie-gram"  to  establish  the  factor 
weights.  The  pie-gram  (the  name  was  arbitrary)  is  a subjec- 
tive measurement  technique  that  employs  a "constant  sum" 
method  [8] . This  method  of  obtaining  the  factor  weights  is 
important  for  two  reasons:  (1)  the  results  of  such  a 
method  achieves  the  level  of  ratio  measurement  [8];  and 
(2)  the  method  represents  a departure  from  theoretical 
purity.  That  is,  the  research  a priori  forces  the  additive 
form  of  the  Multiple  Attribute  Utility  Function  (MAUF)  on 
the  data  collected  since  the  sum  of  the  factor  weights  = 

1.0  (£K^  = 1.0),  and  this,  by  definition,  is  the  additive 
form.  Thus,  even  if  the  multiplicative  form  was  determined 
to  be  the  correct  form,  the  researcher  has  biased  the  data 
in  such  a manner  that  he/she  would  be  unable  to  use  this 
correct  form.  The  reasons  for  such  a priori  biasing  were 


as  follows: 
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1.  This  researcher  feels  that  every  effort  to  insure 


ratio  data  should  be  pursued  by  analysts  attempting  to 

measure  arms  transfers.  The  perils  of  anything  less  were 

clearly  identified  by  Legrow  and  his  rebuttal  of  Mihalka's 
12 

interval  scores.  Thus,  the  respondents  were  not  simply 
asked  to  numerically  weight  the  factor's  overall  contribu- 
tion to  the  process  because  it  was  felt  this  free  form  of 
subjective  measurement,  although  yielding  unbiased  results, 
entails  too  large  an  assumption  for  ratio  measurement.  This 
assumption  entails  a belief  that  enough  regularity  exists 
in  value  judgment  to  measure  them  on  a ratio  level.  Thus, 
by  more  reliable  constant  sum  method,  as  discussed  by 
Torgerson  [7]  was  deemed  a better  approach  to  the  subjective 
weighting  of  factors  at  the  ratio  level. 

2.  The  consequences  of  forcing  the  additive  form  on 
the  generated  data  are  minimal  because  the  measure  of  wea- 
pon systems  are  comparative  and  not  absolute  in  nature 
(this  further  assumes  transformations  between  the  additive 
and  multiplicative  forms  are  consistent).  To  illustrate, 
consider  a hypothetical  situation  where  the  multiplicative 
form  of  the  MAUF  yielded  a score  of  167  and  the  additive 
form  yielded  a score  of  2012.  The  reader  will  immediately 
notice  there  is  a significant  difference  of  one  order  of 
magnitude  between  these  two  scores,  which  is  precisely  the 


12 

See  Chapter  II,  Section  A. 
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point.  The  reader  has  compared  one  against  the  other  and 
deduces  a significant  difference  exists.  If  given  the  value 
of  2012  alone,  how  does  one  ascertain  whether  this  is 
"good"  or  "bad?"  Thus,  as  long  as  the  analyst  computes  the 
scores  in  an  identical  manner,  the  errors  in  an  absolute 
sense  are  irrelevent. 

The  results  presented  in  the  following  chapter 
reflect  the  above  decisions. 


B.  APPLICATION  OF  THE  DATA  TO  THE  F4J  "PHANTOM" 

To  apply  MAUT  to  fighter  aircraft,  the  analyst  must  first 
collect  the  pertinent  factor  information  on  the  weapon 
system  being  studied. 

The  data  for  the  F4J  "Phantom"  were  collected  from  the 
duty  officer  of  an  operational  squadron  at  Naval  Air  Station 
Miramar.  These  data,  if  not  exact,  will  serve  for  illustra- 
tive purposes  and  are  listed  below: 


FACTOR 


F4J  "PHANTOM" 


Dash  Speed 


MACH  1.8 


Acceleration  (T/W) 


. 91:1 


Wing  Loading 


92  lbs/sq  ft 


Combat  Radius 


500  nm 


No.  of  Gun  Barrels 


Missile  Speed 
Missile  Angle-Off 


3.5  (Sparrow) 
360°  (Sparrow) 


Missile  Range 


24  nm  (Sparrow) 


29 


Next,  the  analyst  enters  the  appropriate  utility  curve 


with  the  above  information  and  reads  the  utility  value 
associated  with  a particular  entry.  For  example.  Figure  3 
depicts  the  resultant  dash  speed  utility  curve  (the  other 
factor  utility  curves  are  located  in  Appendix  B)  for  this 
survey.  The  analyst  would  enter  the  graph  on  the  horizontal 
axis  with  the  "Phantom's"  dash  speed  value  (MACH  1.8). 

Next,  a vertical  line  is  drawn  from  this  point  upward  until 
it  intersects  the  utility  curve.  A horizontal  line  is  drawn 
from  this  point  to  the  vertical  axis  and  the  judge's  utility 
rating  for  MACH  1.8  is  read.  In  this  instance,  the  value 
would  be  .94. 

This  procedure  was  applied  to  the  rest  of  the  factors 
and  the  F4J  "Phantom"  scores  were  as  follows: 

FACTOR  F4J  "Phantom"  SCORES 


Dash  Speed  0.94 
Acceleration  0.80 
Wing  Loading  0.40 
Combat  Radius  0.80 
No.  of  Gun  3arrels  0.00 
Missile  Speed  0.86 
Missile  Angle-Off  1.00 
Missile  Range  0.54 


Once  these  factor  scores  are  obtained,  the  analyst  must 
determine  their  combinatorial  relationship  to  each  other. 

As  mentioned  earlier,  this  survey  a priori  forced  the  results 
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into  the  additive  form.  The  properties  are  still  checked, 
however,  to  insure  that  some  form  of  the  MAUF  applies. 

Thus,  the  analyst  checks  which,  if  any,  of  the  following 
properties  hold: 

1.  Pairwise  Marginality 

Pairwise  marginality  is  the  first  property  checked 
because  if  it  holds,  utility  independence  and  pairwise  pref- 
erential independence  are  satisfied  automatically  [5] . 
Pairwise  marginality  presents  the  judge  with  two  alternatives 
and  checks  whether  or  not  the  judge  would  be  indifferent 
between  them.  That  is,  would  the  judge  let  the  toss  of  a 
coin  determine  which  alternative  he  or  she  would  receive. 
These  alternatives  take  two  factors  simultaneously  (eventu- 
ally considering  all  possible  pairs  of  factors)  and  presents 
the  judge  with  a series  of  hypothetical  situations  using 
the  maximum  and  minimum  values  obtained  from  the  respective 
utility  curves.  To  illustrate  this  procedure,  a sample 
case  using  the  aircraft  platform  factors  is  presented. 


Minimum 

Maximum 

S = Dash  Speed 

MACH  .7 

MACH  1 . 9 

A = Acceleration  (T/W) 

.4/1 

1.1/1 

W = Wing  Loading 

140 

19 

R = Combat  Radius 

80  nm 

415  nm 

t 

The  values  for  the  minimum/maximum  are  obtained  from  the 
utility  curves.  These  values  were  chosen  because  extreme 
cases  usually  precipitate  easier  decisions  for  the  judge. 
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Any  values  between  the  minimum  and  maximum  are  permissible. 
Once  the  symbols  are  defined  and  the  minimum  and  maximum 
values  set,  the  judge  is  presented  with  a series  of  lotter- 
ies (choices)  similar  to  the  following. 

Indifferent  between  A and  B 

Case  1 

Let  A = 50%  chance  the  aircraft  Yes  No  

will  have  a S = 1.9  (maximum) 
and  A = 1.1  (maximum)  and  a 50% 
chance  the  aircraft  will  have  a 
S = .7  (minimum)  and  A = .4 
(minimum) 

Let  B = 50%  chance  the  aircraft 
will  have  a S = 1.9  (maximum) 
and  A = .4  (minimum)  and  a 50% 
chance  the  aircraft  will  have  a 
S = .7  (minimum)  and  A = 1.1 
(maximum) 

By  answering  "Yes"  in  the  above  lottery,  the  judge  insinu- 
ates he  would  let  the  toss  of  a coin  determine  which  alterna- 
tive, A or  B,  would  govern  the  factors  for  fighter  aircraft. 

A "No"  answer  implies  the  judge  prefers  one  alternative  to 
the  other . 

For  the  following  cases,  the  same  "50%"  rules  apply. 
The  verbage,  however,  will  be  omitted  and  the  lotteries 
will  have  the  following  format. 


F 


A = 

(S 

= 

1.9,  A 

= 

1.1) 

or 

(S  = .7,  A 

= .4) 

B = 

(S 

= 

1.9,  A 

= 

• 4) 

or 

(S  - .7,  A = 

1.1) 

Yes 

No 

Case  2 

A = 

(s 

= 

1.9,  W 

= 

19) 

or 

(S  = .7,  W = 

140) 

B = 

(s 

= 

1.9,  W 

= 

140) 

or 

(S  = .7,  W 

= 19) 

Yes 

No 

Case  3 

A = 

(S 

= 

1.9,  R 

= 

415) 

or 

(S  = .7,  R 

= 80) 

B = 

(S 

= 

1.9,  R 

= 

80) 

or 

(S  = .7,  R = 

415) 

Yes 

NO 

Case  4 

A = 

(A 

= 

1.1,  W 

= 

19) 

or 

(A  = .4,  W = 

140) 

B = 

(A 

= 

1.1,  W 

= 

140) 

or 

(A  = .4,  W 

= 19) 

Yes 

No 

Case  5 

A = 

(A 

= 

1.1,  R 

= 

415) 

or 

(A  = .4,  R 

= 80) 

B = 

(A 

= 

1.1,  R 

= 

80) 

or 

(A  = .4,  R = 

415) 

Yes 

No 

Case  6 

A = 

(W 

= 

19,  R = 

415) 

or 

(W  = 140,  R 

= 80) 

B = 

(W 

= 

19,  R = 

80)  or  (W  = 140,  R = 

415) 

Yes 

No 

If  the  judge  answered  Yes  for  all  six  cases,  addi- 
tivity has  been  established  for  the  platform  factors  and  the 
additive  form  of  the  multiple  attribute  utility  function  is 
used.  The  same  procedure  is  then  applied  to  the  other  fac- 
tors and  the  binomial  coefficient  provides  a convenient 
formula  for  deducing  the  number  of  cases 

N! 

R! (N-R) ! 

where  N = total  number  of  factors  being  considered;  R = num- 
ber of  factors  being  considered  at  a time.  For  pairwise 
marginality  computations,  R = 2 in  all  cases. 
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The  respondents  cf  this  survey  voted  against  the 


additive  form  of  the  multiple  attribute  utility  function. 
They  felt  alternative  B always  represented  a better  choice 
than  alternative  A.  It  was  felt  the  risk  of  receiving  the 
"minimum-minimum"  factor  combination  possible  with  alterna- 
tive A left  the  pilot  no  chance  to  develop  a successful 
tactic.  Alternative  B,  on  the  other  hand,  always  offered  a 
"minimum-maximum"  factor  combination  and  it  was  felt  a 


tactic  could  be  developed  to  protect  the  weak  factor  and 
exploit  the  strong  one. 

If  the  test  for  pairwise  marginality  fails  at  any 
point,  (which  occurs  with  the  first  "No"  answer)  the  analyst 
must  then  determine  if  the  multiplicative  form  applies.  To 
do  this,  pairwise  preferential  independence  is  considered 
first  as  failure  to  establish  this  property  automatically 
implies  the  multiplicative  form  of  the  utility  function  is 
not  applicable. 

2 . Pairwise  Preferential  Independence 

Using  the  definitions  and  the  minimum/maximum 
values  of  the  pairwise  marginality  example,  the  analyst 
again  presents  the  judge  with  a series  of  lotteries.  This 
time,  however,  the  judge's  lotteries  involve  all  four 
factors  at  once.  Basically,  choice  A fixes  two  factors  at 
their  minimum  values  and  allows  the  other  factors  to  assume 
some  arbitrary  value  between  their  minimum  and  maximum. 
Choice  B takes  the  fixed  values  in  choice  A and  changes  the 
values  to  the  maximum.  The  other  factors  assume  the  same 
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arbitrary  values  assigned  in  choice  A.  The  judge  is  then 
asked  if  his/her  choice  between  A and  B depends  only  on  the 
values  assigned  to  the  fixed  factors.  This  procedure  is 
then  repeated  until  all  possible  pairs  of  factors  have  been 
tested.  If  the  judge's  choice  is  positive  in  all  cases, 
pairwise  preferential  independence  has  been  established. 

This  property,  however,  is  insufficient  to  justify  the 
multiplicative  form  by  itself,  therefore,  utility  indepen- 
dence must  be  checked.  Before  discussing  utility  indepen- 
dence, an  example  of  pairwise  preferential  independence  is 
presented. 

Case  1 

Let  A = (S  = .7,  A = .4,  W = "X",  R = "Y" ) and 

B=  (S  = 1.9,  A = 1.1,  W = "X",  R = "Y") 
the  judge  is  then  asked  if  there  are  any  values  for  "X"  and 
"Y"  (between  the  minimum  and  maximum  values  allotted  by 
their  respective  utility  curves)  that  could  make  him/her 
indifferent  between  lottery  A and  lottery  B.  Those  values 
do  not  have  to  be  ascertained.  It  is  sufficient  to  know 
they  exist.  If  there  are  none,  pairwise  preferential  inde- 
pendence has  been  established  for  these  two  factors.  The 
analyst  must  then  repeat  the  procedure  until  all  possible 
combinations  have  been  exhausted.  All  pairs  must  exhibit 
pairwise  preferential  marginality  to  establish  its  existence. 

For  this  test,  the  respondents  indicated  a preference 
in  each  case,  thereby  establishing  this  property.  The  rea- 
sons were  similar  to  the  ones  expressed  during  the  test  for 
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pairwise  marginality.  That  is,  the  respondents  felt  tactics 
could  be  successfully  developed  better  around  one  alterna- 
tive than  the  other.  With  this  property  established  and 
given  accessible  respondents  (if  the  respondents  are  inacces- 
sible, the  analyst  must  assume  the  questionnaire's  guidance 
in  this  area  was  sufficient)  the  analyst  must  then  reaffirm 
the  existence  of  factor  utility  independence. 

3 . Utility  Independence 

Utility  independence  is  established  when  the  judge 
assures  the  analyst  that  the  values  given  for  any  particular 
utility  curve  do  not  reflect,  depend  on  or  relate  to  the 
values  of  any  other  utility  curve.  In  other  words,  was  the 
respondent  considering  the  potential  trade-offs  between 
attributes,  say,  dash  speed  and  manueverability , while  map- 
ping their  utility  curves?  If  not,  utility  independence  has 
been  established. 

As  a review,  pairwise  marginality  is  required  for 
the  additive  form  of  the  utility  function.  Pairwise  pre- 
ferential independence  and  utility  independence  are  both 
necessary  for  the  multiplicative  form.  If  neither  form  can 
be  established,  Keeney  [16]  suggests  the  additive  form 
serves  as  a good  approximation  to  the  utility  function. 

The  factors  may  be  grouped  in  their  "natural"  setting,  as 
they  were  for  this  experiment,  (i.e.,  dash  speed,  accelera- 
tion, wing  loading  and  combat  radius,  considered  together) 
with  a subsequent  inter-group  check  or  they  may  be  considered 
all  together.  The  decision  is  left  to  the  analyst. 
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In  this  survey,  the  additive  form  was  applied  to 
the  data  for  the  previously  mentioned  reasons.  (Note:  The 
technological  evaluations  for  each  country  are  not  opera- 
tive at  this  point,  therefore,  scores  reflect  only  the 
aircraft,  the  weapons  and  pilot  nationality.) 

The  results  are  as  follows: 

Utility  score  for  Israeli  pilots  (who  have  an 
average  of  1000  hours  flight  time,  while  Egyptians  have  an 
average  of  500  hours)  in  the  F4J  "Phantom"  with  Sparrow 
missile 

9 

= Z K . • U (X . ) = [ (F4J  dash  speed  score  x dash  speed 

1=1  1 13 

weight  ) + (F4J  acceleration  score 

x acceleration  weight)  + ...  + (Israeli 
pilot's  score  x pilot  weight)] 

= [(.94  x .12)  + (.8  x .14)  + ...  + (.6  x .31)] 

= .65 

Utility  score  for  Egyptian  pilots  in  F4J  "Phantom"  with 
Sparrow  missile 
9 

= Z K. -U(X. ) = .53. 
i=l  1 1 

Thus,  if  one  assumes  equal  logistical  support,  the 
Egyptians  need  1.22  F4Js  for  every  one  of  the  Israelis'  to 
achieve  analytical  parity.  With  the  appropriate  aircraft 
data,  the  analyst  could  evaluate  each  aircraft  in  the 

^A  list  of  the  resultant  factor  weights  with  a brief 
discussion  may  be  found  in  Appendix  C. 
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respective  inventories  to  determine  each  country's  air-to- 
air  capability.  Once  established,  future  foreign  requests 
for  arms  could  be  evaluated  against  these  analytical 
parity  checks. 

In  the  next  chapter,  the  theory  is  gently  stretched 
in  an  effort  to  increase  the  experiment's  sample  size. 

This  is  important  because  small  sample  sizes  (e.g.,  4,  8, 

12,  20)  are  (statistically)  insufficient  to  establish  a 
high  degree  of  confidence  in  the  results.  There  is  virtually 
no  way  of  verifying  whether  the  four  respondents  (for  this 
survey)  represent  the  norm  or  the  extreme.  This  is  not  a 
problem  peculiar  to  multiple  attribute  utility  theory,  it 
is  a problem  that  arises  whenever  outside  (non-textbook) 
data  is  fed  through  an  analytical  formula.  A large  sample 
size,  therefore,  is  a simple  but  effective  way  to  help 
alleviate  the  "garbage  in-garbage  out"  syndrome  so  often 
associated  with  the  real  world/theoretical  model  combinations. 

C.  LESSONS  LEARNED 

The  difficulties  encountered  with  this  questionnaire 
centered  on  the  utility  curves  and  their  derivations. 
Originally,  respondents  were  solicited  from  the  Naval  Post- 
graduate School  and  the  fighter  squadrons  located  at  the 
Naval  Air  Station  Miramar.  The  respondents  solicited  at 
Miramar  did  not  have  any  prior  knowledge  of  utility  curves 
or  their  uses.  This  was  not  by  design,  it  was  more  or  less 
a function  of  the  circumstances  that  brought  the  examiner 
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and  the  respondents  together.  The  explanation  of  utility 
curves  was  not  well  received  and  the  seven  returned  ques- 
tionnaires (out  of  a possible  30)  reflected  the  anticipated 
misperceptions.  Thus,  the  sample  size  was  reduced  to  the 
four  aviators  at  the  Naval  Postgraduate  School  who  possessed 
a knowledge  of  utility  curves  and  their  uses. 

It  is  felt  that  further  classical  research  should  con- 
fine its  respondents  to  those  with  prior  knowledge  of 


utility  curves  and  their  uses. 


This  chapter  presents  an  alternative  to  a data  collec- 
tion procedure  cited  in  Chapter  III.  Instead  of  the 
required  utility  curves,  respondents  would  be  instructed  to 
identify  three  points  for  each  factor  — the  lower  limit, 
the  upper  limit,  and  the  50th  utile.  These  points  are  iden- 
tical to  their  counterparts  in  the  factor  utility  curves 
and  would  therefore  retain  the  relevant  properties  necessary 
for  ratio  measurement.  Once  the  analyst  has  acquired  these 
points,  they  would  be  connected  to  form  a piecewise  linear 
approximation  of  the  actual  utility  curve.  As  an  example, 
consider  a hypothetical  case  where  the  dash  speed  of  a 
fighter  aircraft  is  reported  to  have  a lower  limit  of  MACH 
1.3,  an  upper  limit  of  MACH  2.4  and  a 50th  utile  of  MACH  1.7. 
The  piecewise  linear  approximation  would  appear  as  follows: 

1.0  - 

Respondent ' s 

Utility  .5 

Scale 

0.0 

1.3  1.7  2.4 

MACH  No. 

Admittedly,  the  precision  of  an  actual  utility  curve  is 
lost.  It  is  felt,  however,  the  benefits  of  such  a system 
far  outweigh  the  disadvantages.  These  benefits  are  believed 
to  be:  (1)  The  procedure  will  significantly  broaden  the 
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usable  data  base.  The  vast  supply  of  aviators  yields  rela- 
tively few  with  utility  theory  training.  Under  the  present 
system  of  data  collection,  this  expertise  would  be  lost. 

With  the  three  point  system,  all  aviators  are  potential 
respondents  since  these  values  can  be  ascertained  from  the 
"no  prior  knowledge  necessary"  type  question;  { 2 ) with  a 
small  sample  size,  it  is  extremely  difficult  (statistically) 
for  the  analyst  to  discern  if  the  respondents  reflect  the 
norm  or  the  extreme  cases;  (3)  measuring  the:  capability 
shifts  contingent  to  an  arms  transfer  is  an  inherently  diffi- 
cult task  as  there  are  many  intangibles  attendant  to  such 
transactions.  Therefore,  quantitative  techniques  to  evaluate 
such  transfers  are  approximations  at  best.  Thus,  the  small 
amount  of  precision  lost  in  the  three  point  system  is  not 
considered  critical  by  the  author. 

Section  A discusses  the  questionnaire  used  to  elicit  the 
data.  Section  B analyzes  these  data  in  accordance  with  the 
procedures  outlined  in  Chapter  III.  Section  C summarizes 
respondents'  comments  in  a "lessons  learned"  format. 

A.  DATA  COLLECTION 

A four  part  questionnaire  was  the  source  of  data  acqui- 
sition for  this  experiment.  Part  one  establishes  the  lower 
and  upper  limits  for  each  factor.  The  verbage  was  non- 
technical and  omitted  any  reference  to  utility  curves  and 
their  underlying  concepts.  This,  of  course,  was  designed  to 
permit  respondents  with  little  or  no  utility  theory  training 
or  education  to  respond.  The  lower  limit  was  intended  to 
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represent  the  point  of  zero  utility  and  the  upper  limit  the 
point  of  maximum  utility.  Additionally,  instructions 
designed  to  (hopefully)  insure  utility  independence  were 
included,  similar  to  the  effort  in  the  small  sample  survey. 

Part  two  attempts  to  establish  the  50th  utile  for  each 
factor.  This  was  the  most  difficult  section  to  write  as 
the  50th  utile  is  a counter-intuitive  idea  to  those  with  no 
training  in  this  area.  The  50th  utile  may  or  may  not 
represent  the  "half-way"  point  between  the  upper  and  lower 
limits  of  any  particular  factor.  It  all  depends  on  the 
judge  and  his  subjective  opinions  of  performance  versus 
utility.  The  questionnaire  is  located  in  Appendix  D for  the 
reader's  scrutiny.  Part  three  of  this  questionnaire  presents 
a "pie-gram"  identical  to  the  one  employed  by  the  small 
sample  survey.  Thus,  the  data  is  pre-biased  toward  the  addi- 
tive form  of  the  MAUF.  Part  four  was  a confidence  check  by 
the  researcher  to  establish  the  respondents'  perceptions  of 
how  well  they  felt  they  understood  the  questions.  They  were 
asked  to  rate  each  section  on  a scale  of  1-100  (100  being 
the  best)  on  the  basis  of  their  confidence  in  understanding 
the  instructions.  Respondent  questionnaires  that  indicated 
a confidence  level  of  less  than  60%  (arbitrarily)  were 
discarded . 

B.  APPLICATION  OF  THE  DATA  TO  THE  F4J  "PHANTOM" 

The  vehicle  used  for  analyzing  the  data  was  the  histo- 
gram [8].  It  was  felt  the  pictorial  display  of  the  data's 
distribution  provided  the  best  tool  to  establish  the  single 
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value  necessary  for  each  limit.  The  particular  computer 
program  used  (FORTRAN  HISTF)  to  generate  the  graphics 
additionally  features  (1)  a sample  density  function  super- 
imposed over  the  data  cells,  and  (2)  twenty-six  statistical 
measures  of  data  location,  dispersion  and  distributional 
characteristics.  Figure  4 displays  the  histogram  analysis 
of  the  data  received  for  the  lower  limit  of  dash  speed  (the 
remainder  of  the  graphics  are  located  in  Appendix  E) . The 
single  value  decided  upon  for  each  factor's  limit  was  the 
sample  mean.  This  value  was  chosen  because  (1)  the  differ- 
ence among  the  measures  of  location  (mean,  median,  trimean, 
etc.)  are  minimal,  and  (2)  there  are  a variety  of  analytical 
devices  (provided  the  sample  is  large  enough  to  invoke  the 
central  limit  theorem)  to  establish  the  sample  mean's  relia- 
bility relative  to  the  true  population  mean. 

To  illustrate,  consider  the  sample  mean  for  the  lower 

14 

limit  of  dash  speed  (MACH  1.36).  Using  the  T-test  , the 
analyst  could  easily  establish,  say,  a .90  confidence 

interval  for  the  population  mean.  The  appropriate  equation 

. 15 
is 


X - t 


.95,199 


< M < X + t 


.95,199 


= 1.36  - 1.282  • -^1  < v < 1-  36  + 1.282 


. 421 
/TOO 


= [1.321,  1.398] 


^reund,  John  E. , Mathematical  Statistics,  p.  274, 
Prentice-Hall,  Inc.,  2nd  Edition,  1971. 


Ibid. , p.  274 
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•05  tail  area  of  a 


where  X is  the  sample  mean,  t 


.95,199 


T distribution  with  199  degrees  of  freedom,  S is  the  sample 
standard  deviation,  N is  the  sample  size  and  y is  the 
population  mean. 

Thus,  we  are  90%  confident  that  the  true  population 
mean  falls  somewhere  between  MACH  1.321  and  MACH  1.398.  The 
same  procedure  is  easily  applied  to  the  other  points.  From 
a practical  standpoint,  the  policy  maker  can  be  assured  his 
evaluative  model  gives  (statistically)  reliable  results. 

Additionally,  a scattergram  [10,11]  of  the  respondents' 
pilot  hours  versus  the  factor  limits  and  weights  was  compu- 
ted to  discern  if  there  was  any  correlation  between  the  total 
number  of  hours  accumulated  by  a fighter  pilot  and  his 
response  to  the  questions.  Figure  5 displays  the  scatter- 
gram of  pilot  hours  versus  the  lower  limit  for  dash  speed. 

The  correlation  for  the  sample  regression  line  was  0.04981, 
in  this  case  indicating  an  insignificant  relationship 
between  pilot  hours  and  what  was  considered  optimal  for  the 
lower  limit  of  dash  speed.  The  remaining  scattergrams  for 
this  experiment  are  located  in  Appendix  E along  with  the 
histograms . 

A table  with  the  resultant  values  for  the  lower  limit, 
the  upper  limit,  the  50th  utile  and  the  factor  weighting  are 
presented  below  (their  resultant  utility  curves  are  located 
in  Appendix  F) . Additionally,  the  new  scores  (as  opposed 
to  the  ones  calculated  from  the  small  sample  survey)  for 
the  F4J  "Phantom"  equipped  with  the  Sparrow  missile  are 
included . 
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HOURS 


Limit  for  Dash  Speed 


FACTORS 

LOWER 

LIMIT 

50TH 

UTILE 

UPPER 

LIMIT 

WEIGHT 

F4J 

SCORE 

Dash  Speed 

1.36 

1.8 

2.4 

.109 

.50 

Acceleration 

(T/W) 

.91 

1.2 

1.6 

.141 

.01 

Wing  Loading 

69.00 

55.1 

40.3 

.134 

0.00 

Combat 

Radius 

315.00 

452.7 

648.8 

.085 

.62 

No.  of  Gun 
Barrels 

3.44 

5.2 

7.5 

.052 

0.00 

Missile 

Speed 

2.00 

2.6 

3.8 

.059 

.88 

Missile 

Angle-Off 

77.50 

107.9 

182.4 

.140 

1.00 

Missile  Range  5.10 

15.7 

32.6 

.075 

.74 

Pilot  Hours 

533.90 

830.2 

1486.0 

.249 

.63(1) 
0.00 (E) 

Reiterating  the  procedures 

outlined 

in  Chapter  III,  the 

additive  form  of  the 

large  sample  data  yields  the 

following 

results : 

Utility 

score  for 

Israeli 

(I)  pilots 

in  F4J  " 

Phantom " 

9 

= Z 
L= 

KrU(X 

1 1 1 

) = .512 

Utility 

score  for 

Egyptian 

(E)  pilot 

in  F4J  " 

Phantom" 

9 

= Z K. -U(X. ) = .356 
L=1  1 1 

Thus,  the  large  sample  survey  suggests  the  Egyptians''-6  need 
1.43  F4J  "Phantoms"  to  every  1.0  the  Israelis  have  to  achieve 
analytical  parity. 


16Again,  in  the  absence  of  data,  the  Egyptian  and 
Israeli  logistical  efforts  are  assumed  equal. 
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One  of  the  expressed  purposes  of  the  large  sample  survey 
was  to  validate  the  proposed  three  point  utility  curve  esti- 
mator as  a statistically  reliable  tool.  Unfortunately,  the 
resultant  sample  size  of  the  small  sample  survey  (four)  was 
insufficient  to  apply  the  available  statistical  tests  to 
discern  if  the  differences  between  the  two  samples  were 
statistically  significant.  Perhaps  a future  research  effort 
could  gather  a larger  sample  of  utility  curves  for  these 
factors  and  compare  the  results  with  the  large  sample  data 
presented  in  this  thesis. 

If  one  were  to  view  the  results  from  a non-technical 
approach,  however,  the  differences  between  1.22:1  (the 
results  from  the  small  sample  survey  implying  the  Egyptians 
require  1.22  F4J  "Phantoms"  to  every  1.0  the  Israelis  pos-  '■ 
sess  to  maintain  parity)  and  1.43:1  (results  from  the  large 
sample  survey)  are  quite  minimal.  In  fact,  considering  the 
gross  difference  in  sample  sizes,  the  results  are  surprisingly 
close.  Perhaps  this  is  suggestive  that  the  respondents  in 
both  surveys  represent  identical  populations  and  value 
differences  are  merely  a function  of  the  sample  size.  If 
these  observations  are  considered  judicious  the  analyst  can 
conclude  the  three  point  utility  curve  estimators  are  reli- 
able inputs  to  the  multiple  attribute  utility  function. 

C.  LESSONS  LEARNED 

The  majority  of  responses  indicated  a dissatisfaction 
with  three  factors  — wing  loading,  number  of  gun  barrels, 
and  missile  range.  It  was  felt  that  fusilage  lift  and  a 
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host  of  aerodynamic  changes  rendered  wing  loading  invalid  as 
an  indicator  of  a modern  aircraft's  turning  radius.  The 
number  of  gun  barrels  was  generally  received  with  multiple 
question  marks  and  comments  suggesting  the  one  gun-one  bar- 
rel concept  has  been  replaced  by  the  Gatling  gun.  Missile 
range,  it  seems,  should  have  been  presented  in  two  cate- 
gories, missile  range  "close  in"  and  missile  range  "distant." 

Perhaps  the  comments  indicate  that  measures  of  aircraft 
(and  other  weapon  systems)  are  fluid  in  nature.  No  single 
"once-and-for-all"  measure  of  capability  lurks  about  waiting 
to  be  discovered.  Accurate  evaluation,  by  any  means, 
requires  a continual  updating  of  the  factors  surrounding  a 
weapon  system. 


V.  CONCLUSIONS 


The  desire  to  accurately  evaluate  the  transfer  of  arms 
to  lesser  developed  countries  is  a growing  concern  among 
many  military  planners.  Various  quantitative  techniques 
have  been  employed  to  measure  these  transfers  but  to  date, 
none  have  produced  a reliable  indicator.  This  thesis  examines 
Multiple  Attribute  Utility  Theory  as  a possible  alternative. 
The  study  has  consumed  one  year  and  concludes  with  the  con- 
viction that  multiple  attribute  utility  theory  is  a signifi- 
cant step  forward  in  the  search  for  a valid  measure  of  arms 
transfers.  The  advantages  enjoyed  by  those  who  would  employ 
this  methodology  are  perceived  to  be  as  follows: 

1.  MAUT  decomposes  weapon  systems  into  their  basic 
elements.  These  elements  are  classless,  measureable  and  eas- 
ily understood.  Every  fighter  aircraft  has  a dash  speed  and 
although  experienced  aviators  may  not  be  able  to  specify 
what  the  optimum  value  should  be,  they  can  bracket  it. 

2.  MAUT  establishes  the  weight  or  relative  importance 
for  each  of  these  elements.  Thus,  different  permutations 
of  pilots,  weapons  and  aircraft  are  easily  calculated. 

3.  The  input  data  for  this  device  is  derived  from  judges 
with  operational  expertise.  It  is  not  a tool  wielded  by 
those  who  have  never  seen,  sat  in  or  operated  the  weapon 
system  they  are  about  to  judge. 
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4.  As  more  factors  become  relevant  or  capable  of  meas- 


urement (e.g.,  technological  level  of  the  country),  the 
theory  allows  "newcomers"  to  be  included  into  the  calculations. 

5.  With  two  minor  assumptions,  MAUT  achieves  the  level 
of  ratio  measurement.  Only  with  ratio  measurement  can  one 
say  "aircraft  A is  1.4  times  better  than  aircraft  B." 

6.  It  circuitously  quantifies  a host  of  "intangibles." 

The  "intangibles"  are  an  important  part  of  any  man-machine 
interface  yet  most  analysts  would  be  hard  pressed  to  define 
or  quantify  these  factors.  MAUT  solves  this  problem  by 
having  pilots  evaluate  pilots,  tank  drivers  evaluate  tank 
drivers,  etc.,  in  a common  denominator  — hours  of  experience. 
These  people  may  not  be  able  to  verbalize  their  thoughts  and 
feelings  about  what  makes  a good  pilot  or  tank  driver,  but 
they  are  considered  when  the  analyst  asks  them  "What  is  the 
minimum  number  of  hours  you  want  your  wingman,  etc. , to 
have?" 

The  disadvantages  of  MAUT  are: 

1.  The  properties  that  govern  what  form  of  the  utility 
function  to  use,  i.e.,  how  to  combine  the  factors,  are 
difficult  to  establish.  It  is  suggested  this  is  not  as 
critical  a point  as  it  might  first  appear.  The  measurement 
of  weapon  systems  are  inherently  comparative  in  nature.  It 
would  seem  decision  makers  are  not  concerned  with  how  good 
a weapon  is  absolutely,  but  rather  how  good  is  it  compared 
to  what  their  opponents  have.  Thus,  the  small  errors  that 
arise  from  using  the  incorrect  form  are  irrelevant  as  long 
as  the  same  form  is  used  for  all  computations. 
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2.  The  utility  curves  required  of  each  judge  serve  to 
limit  the  sample  size  of  any  experiment  and  small  sample 
experiments  are  always  suspect.  With  a minor  modification, 
however,  this  problem  is  alleviated  and  sample  size  could 
potentially  reflect  the  entire  class  of  experienced  operators. 

3.  The  implications  of  aggregating  responses  (i.e., 
utility  curves  and  factor  weights)  have  not  been  fully  deci- 
phered. Future  research  should  validate  this  point  before 
MAUT  is  implemented  as  a decision  making  tool. 

During  the  course  of  this  research,  some  interesting 
observations  were  made. 

1.  The  descriptive  factors  for  fighter  aircraft  used  by 
this  study  were  established  by  previous  research.  It  seems 
a reasonable  conclusion  they  were  the  accepted  norm  at  the 
time,  yet,  a majority  of  the  large  sample  respondents  indi- 
cated a dissatisfaction  with  these  factors.  Perhaps  this  is 
an  indication  that  norms  or  measures  of  military  worth  are 
not  fixed  but  evolutionary  in  nature. 

t 

2.  If  MAUT  is  not  considered  a judicious  measurement 
technique,  within  its  framework  lies  a potentially  useful  tool. 
The  "tool"  is  the  knowledge  of  what  fleet  personnel  perceive 

to  be  good.  The  respondents  to  the  large  sample  survey  may 
have  chosen  poor  values  for  each  factor,  but  those  values, 
right  or  wrong,  reflect  their  perceptions  of  what  is  "right." 

It  would  seem  a knowledae  of  the  fighting  man's  perceptions 
are  useful  data. 
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APPENDIX  A 


CHARACTERISTICS 

OF 

FIGHTER  AIRCRAFT 

The  ever  increasing  transfer  of  fighter  aircraft  to  the 
lesser  developed  countries  generates  a mounting  concern 
among  many  military  planners.  The  current,  quantitative 
techniques  to  evaluate  the  military  worth  of  such  trans- 
fers often  revolve  around  the  budgetary  or  inventory 
type  models.  This  research  effort  intends  to  expand  on 
the  inventory  method  by  including  such  factors  as  the 
maintenance  capability  of  the  country,  pilot  experience 
and  the  previously  untested  concept  of  comparing  their 
fighters  against  the  "ideal"  fighter,  a (theoretical) 
plane  designed  from  the  experience  of  many  pilots  in  the 
various  TACAIR  communities  that,  hopefully,  is  captured 
by  this  questionnaire. 

Thus,  to  complete  this  survey,  you  need  not  remember 
specific  technical  features  of  any  particular  aircraft. 
Just  respond  using  your  intuition  and  experience,  and 
remember  the  survey  is  concerned  with  air-to-air  perform- 
ance factors.  Please  work  independently  as  cross  compari- 
sons with  other  participants  will  invalidate  the  subse- 
quent mathematical  tests. 
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TOTAL  PILOT  HOURS 
BRANCH  OF  SERVICE 


Below  are  a number  of  factors  describing  aircraft  perform- 
ance. Previous  research  has  identified  these  characteristics 
as  a reasonably  complete  way  of  classifying  aircraft.  Your 
task  for  this  section  is  to  draw  a utility  curve  for  each 
factor,  keeping  in  mind  this  survey  is  only  concerned  with  the 
factor's  relationship  in  aerial  (air-to-air)  combat.1  Each 
curve  should  begin  at  the  point  of  zero  utility  and  at  least 
pass  through  the  point  of  maximum  utility.  The  vertical  scale 
represents  "utility"  and  is  annotated  from  0 - 1.0,  1.0  repre- 
senting maximum  utility.  The  horizontal  scale  represents  the 
factor  and  is  annotated  in  the  appropriate  units. 

When  considering  these  factors  and  their  utility  curves, 
try  to  consider  each  one  individually,  ignoring  its  effect  or 
relation  to  any  other  factor.  Admittedly,  this  is  not  possible 
in  the  real  world  for  one  factor  (say  manueverability)  is 
always  considered  in  terms  of  other  factors  (i.e.,  wing  loading, 
top  speed,  etc.)  but  for  now,  assume  we  have  some  magic  process 
that  can  incorporate  one  factor  at  no  consequence  to  another. 


Consider  the  aircraft  in  a combat  configuration  in  day,  VFR 
conditions . 
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AERIAL  COMBAT 
UTILITY 


.4 


.2 


THRUST/WEIGHT 


I 

A 


AERIAL  COMBAT 
UTILITY 


.6 


. 4 


.24- 


missile  SPEED  (MACH  No.) 


AERIAL  COMEAT 
UTILITY 


SECTION  II 


In  this  section  you  are  to  gauge  the  relative  importance 
of  each  factor  in  its  contribution  to  overall  air-to-air  combat 
superiority.  The  manner  in  which  you  will  indicate  your  pre- 
ferences is  with  the  "pie-gram."  The  "pie-gram"  is  a pictorial 
way  of  indicating  the  importance  of  a factor  by  the  thickness 
or  thinness  of  the  slice.  This  test  is  often  used  when  there 
are  many  factors  for  it  is  generally -easier  to  judge  the  rela- 
tive importance  when  you  can  "see"  the  size  of  one  slice  com- 
pared to  another.  Pies  are  annotated  from  0 - 100  to  facilitate 
the  divisions  and  provide  later  analysis. 

As  an  example,  consider  a race  car  and  assume  that  three 
factors  — car,  driver  and  track  — completely  describe  the 
elements  necessary  to  win  a race.  A piegram  could  look  like 
the " following: 


is  the  most  important  factor  for  winning  the  race.  Its  portion 
takes  up  60%  of  the  overall  pie  and  is  three  times  as  large  as 
any  other  "slice." 

Our  "pie"  will  have  nine  (09)  slices  as  there  are  nine  per- 
formance factors.  Please  divide  the  "pie"  in  accordance  with 
the  above  example. 

FACTORS 

Speed 

Acceleration 


Wing  Loading 
Combat  Radius 


Number  of  Gun  Barrels 


As  you  divide  the  "pie"  please  number  each  slice  and  place  the 
corresponding  number  in  the  appropriate  blank. 


Missile  Speed 

Missiles  Angle  Off  Capability 
Missile  Range 
Pilot  Experience 
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APPENDIX  C 


SMALL 

SAMPLE  FACTOR  (COMPONENT) 

WEIGHTS 

C-RADIUS 

NO.  OF  GB 

RESP. 

A/C  SPEED 

A/C  ACCEL  WING  LOAD 

1 

.1 

.2 

.1 

.07 

.05 

2 

.05 

.2 

.1 

.1 

.05 

3 

. 025 

.1 

.15 

.025 

.05 

4 

.10 

.05 

. 10 

.05 

. 025 

5 

.10 

.15 

. 15 

.05 

. 075 

6 

. 05 

. 25 

.20 

.10 

.05 

7 

.10 

. 05 

.10 

.10 

. 05 

8 

. 08 

.125 

. 15 

.125 

. 05 

9 

. 30 

. 015 

.25 

. 09 

. 0 

10 

.215 

. 05 

. 14 

.05 

.05 

11 

.20 

.375 

.115 

.05 

.025 

X 

.12 

. 1423 

.1414 

. 0736 

.0382 

a2 

. 00698 

. 01163 

. 0023 

. 0001 

. 0002 

a 

. 0835 

. 1079 

. 048 

. 031 

. 015 
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RESP 


MISS.  SPD  MISS 


-OFF  MISS  RANGE  PILOT  HOURS 


1 

. 05 

. 2 

. 08 

.15 

2 

.05 

.1 

.05 

.30 

3 

.015 

.025 

.01 

.60 

4 

. 025 

.10 

.05 

.50 

5 

. 05 

.10 

.075 

.25 

6 

.05 

.10 

.05 

.15 

7 

.05 

.10 

.05 

.40 

8 

.04 

.08 

.05 

.30 

9 

.015 

. 015 

.015 

.30 

10 

. 05 

.07 

. 05 

.135 

11 

.025 

. 025 

.05 

.135 

X 

. 0382 

. 08  3 

. 048 

. 31 

a2 

.0002 

. 003 

. 0004 

. 021 

a 

. 0150 

. 052 

. 021 

.146 

DISCUSSION 


Examination  of  the  data  reveals  that  the  factor  entitled 
"C-Radius"  (combat  radius)  possesses  the  smallest  variance 
and  therefore  exhibits  the  least  amount  of  data  dispersion 
about  the  mean  (X) . This  infers  chere  was  considerable 
agreement  among  the  respondents  concerning  the  optimum  value 
for  combat  radius.  Thus,  the  analyst  would  have  the  more 
confidence  in  this  value  than  in  any  of  the  others. 

Similarly,  the  factor  labelled  "Pilot  Hours"  has  the 
largest  variance  among  the  received  responses.  This  would 
indicate  the  most  controversial  optimum  value  among  the 
factors . 

Perhaps  the  agreement  exhibited  for  combat  radius 
reflects  the  fact  that  this  factor  was  not  as  important  as 
the  others.  That  is,  the  respondents  collectively  felt  this 
was  a non-critical  factor  and  a value  of  "X"  was  close 
enough.  On  the  other  hand,  pilot  hours  are  more  personal.  A 
small  weighting  of  this  factor  would  imply  a pilot's  part 
is  relatively  minor/unimportant  to  the  overall  mission  and 
vice  versa.  Therefore,  this  research  assumes  the  answers 
will  reflect  the  mental  homogeneity  of  the  respondents  more 
than  objective  observation  of  this  factor's  overall 
contribution. 
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APPENDIX  D 


CHARACTERISTICS 

OF 

FIGHTER  AIRCRAFT 

The  ever  increasing  transfer  of  fighter  aircraft  to  the 
lesser  developed  countries  generates  a mounting  concern 
among  many  military  planners.  The  current,  quantitative 
techniques  to  evaluate  the  military  worth  of  such  trans- 
fers often  revolve  around  the  budgetary  or  inventory 
type  models.  This  research  effort  intends  to  expand  on 
the  inventory  method  by  including  such  factors  as  the 
maintenance  capability  of  the  country,  pilot  experience 
and  the  previously  untested  concept  of  comparing  their 
fighters  against  the  "ideal"  fighter,  a (theoretical) 
plane  designed  from  the  experience  of  many  pilots  in  the 
various  TACAIR  communities  that,  hopefully,  is  captured 
by  this  questionnaire. 

Thus,  to  complete  this  survey,  you  need  not  remember 
specific  technical  features  of  any  particular  aircraft. 
Just  respond  using  your  intuition  and  experience,  and 
remember  the  survey  is  concerned  with  air-to-air  perform- 
ance factors.  Please  work  independently  as  cross  compari- 
sons with  other  participants  will  invalidate  the  subse- 
quent mathematical  tests. 
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SECTION  I 


Total  Pilot  Hours 


Aircraft  Flown 


Branch  of  Service 


Below  are  a number  of  factors  describing  aircraft  perform- 
ance. Previous  research  has  identified  these  characteristics 
as  a reasonably  complete  way  of  classifying  aircraft.  Your  task 
for  this  section  is  to  estimate  a lower  and  upper  limit  for 
each  factor  listed.  The  lower  limit  represents  the  value  (or 
amount)  you  feel  is  the  minimum  a fighter  aircraft  could  have 
and  still  be  effective  in  aerial  combat.  The  upper  limit  is  a 
technologically  feasible  point  that  you  feel  yields  a clearcut 
combat  superiority,  i.e.,  any  further  improvements  would  be  of 
marginal  value. 

For  example,  if  you  were  considering  the  purchase  of  a race 
car,  you  might  choose  a lower  limit  of  140  mph  and  an  upper 
limit  of  250  mph.  By  this  choice  of  limits,  you  feel  that  if  a 
car  (when  considering  the  factor  of  speed  alone)  could  not  do 
at  least  140  mph,  it  would  not  be  in  the  class  of  race  cars  and 
you  would  not  buy  it.  Similarly,  the  upper  limit  of  250  mph 
means  you  feel  this  car  will  easily  outrun  its  competition  and 
further  speed  increases  would  be  of  marginal  value. 

When  considering  these  factors,  try  to  consider  each  one 
individually,  ignoring  its  effect  or  relation  to  any  other  factor. 
Admittedly,  in  the  real  world  this  is  not  possible,  for  an 
increase  or  decrease  in  one  factor  (say  manueverability ) will 
have  some  very  direct  effect  on  other  factors  (i.e.,  top  speed, 
wing  loading,  etc.).  But  for  now,  as  you  mark  the  factors 
below,  assume  we  have  some  magic  process  that  can  alter  one 
factor  at  no  consequence  to  another. 


LOWER  LIMIT  UPPER  LIMIT 

PLATFORM  FACTORS 

a.  Dash  Speed  (MACH  No.)  

combat  configuration,  minimum 
combat  package* 


b.  Acceleration  (thrust/weight  ratio) 
minimum  combat  package* 

c.  Wing  Loading  (lbs/square  foot) 

d.  Combat  Radius  (nautical  miles) 


* Since  the  amount  of  fuel  will  affect  your  answer,  this  require- 
ment is  an  attempt  to  standardize  the  replies.  Here,  the  minimum 
combat  package  means  the  minimum  fuel  state  at  which  you  would 
consider  a combat  engagement  with  a return  to  home  base. 
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SECTION  I (con't) 


LOWER  LIMIT  UPPER  LIMIT 

WEAPON  FACTORS 

a.  Number  of  Gun  Barrels  


b.  Missiles  (do  not  categorize 

mentally  as  IR,  Radar  or  Laser. 
What  feasible  end  results  would 
you  like  to  see) 

(1)  Speed  (MACH  No.) 

(2)  Angle  Off  (degrees) 

(3)  Range  (nautical  miles) 


EXPERIENCE  FACTORS 

Total  (actual)  Pilot  Hours 

Lower  limit  - minimum  number  of 

hours  you  would  want 
your  wingman  to  have 

Upper  limit  - amount  of  (attainable) 
hours  you  wish  your 
wingman  had 
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SECTION  II 


For  this  section,  assume  our  magic  aircraft  engineer  has 
completely  designed  an  aircraft  except  for  one  factor.  For 
this  factor  he  can  choose  a "risky"  design  or  a "safe"  design. 

If  he  picks  the  "risky"  design,  there  is  a 50%  chance  the 
factor  will  have  the  lower  limit  you  specified  (in  Section  I) 
and  a 50%  chance  it  will  have  the  upper  limit. 

On  the  other  hand,  if  the  engineer  chooses  the  "safe" 
design  you  are  100%  certain  the  factor  will  have  some  arbitrary 
"safe  value"  (this  value  would  be  some  number  between  the  upper 
and  lower  limit) . Your  task  is  to  estimate  how  low  this  "safe 
value"  would  have  to  be  before  you  would  consider  giving  it  up 
and  accepting  the  results  of  the  "risky"  design. 

To  further  illustrate,  consider  the  race  car  example  of 
Section  I and  the  fact  3r  of  speed.  The  limits  were  given  as 
140  and  250  mph,  therefore  the  alternatives  would  be  as  follows: 

Risky  Design  - 50%  chance  car  will  have  a speed  of  140  mph 

50%  chance  car  will  have  a speed  of  250  mph 

Safe  Design  - 100%  chance  car  will  have  the  "safe  value" 

If  the  designer  told  you  the  "safe  value"  was  249  mph  and 
asked  you  to  recommend  a design,  you  probably  would  choose  the 
safe  design.  Not  much  point  in  giving  up  a sure  249  mph  for 
what  you  could  get  from  the  risky  design  — a 50%  chance  the 
speed  would  be  250  mph  (1  mph  faster)  and  a 50%  chance  the 

speed  would  be  140  mph  (109  mph  slower!). 

On  the  other  hand,  if  the  designer  told  you  the  "safe  value" 
was  143  mph  and  asked  for  your  recommendation,  you  would  proba- 
bly tell  him  to  go  ahead  with  the  "risky"  design.  Not  much  to 
lose  (going  from  143  mph  down  to  140  mph)  and  a lot  to  gain 
(going  from  143  mph  up  to  250  mph) . So  in  this  case  you  would 
probably  disregard  the  sure  143  safe  value  and  gamble  on  the 
"risky"  design. 

Somewhere  between  the  upper  and  lower  limits  you  selected, 
there  is  a "safe  value"  that  makes  choosing  the  "risky"  design 
just  as  attractive  as  choosing  the  "safe"  design.  Thus,  for 
each  factor  listed  below,  please  indicate  the  cutoff  "safe 
value . " 
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P 


SAFE  VALUE 


PLATFORM  FACTORS 

a.  Dash  Speed  (MACH  No.) 
combat  configuration,  minimum 
combat  package 

b.  Acceleration  (thrust/weight  ratio) 
minimum  combat  package 

c.  Wing  Loading  (lbs/square  foot) 

d.  Combat  Endurance  (nautical  miles) 


WEAPON  FACTORS 


a.  Number  of  Gun  Barrels 

b.  Missiles  (do  not  categorize 
mentally  as  IR,  Radar  or  Laser. 
What  feasible  end  results  would 
you  like  to  see) 

(1)  Speed  (MACH  No.) 

(2)  Angle  Off  (degrees) 

(3)  Range  (n?utical  miles) 

EXPERIENCE  FACTORS 
Total  (actual)  Pilot  Hours 
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SECTION  III 


In  this  section  you  are  to  gauge  the  relative  importance 
of  each  factor  in  its  contribution  to  overall  air-to-air  combat 
superiority.  The  manner  in  which  you  will  indicate  your  pre- 
ferences is  with  the  "pie-gram."  The  "pie-gram"  is  a pictorial 
way  of  indicating  the  importance  of  a factor  by  the  thickness 
or  thinness  of  the  slice.  This  test  is  often  used  when  there 
are  many  factors  for  it  is  generally  -easier  to  judge  the  rela- 
tive importance  when  you  can  "see"  the  size  of  one  slice  com- 
pared to  another.  Pies  are  annotated  from  0 - 100  to  facilitate 
the  divisions  and  provide  later  analysis. 

As  an  example,  consider  the  race  car  mentioned  in  Section  I 
and  assume  that  three  factors  — car,  driver  and  track  — com- 
pletely describe  the  elements  necessary  to  win  a race.  A pie- 
gram  could  look  like  the  following; 


is  the  most  important  factor  for  winning  the  race.  Its  portion 
takes  up  60%  of  the  overall  pie  and  is  three  times  as  large  as 
any  other  "slice." 

Our  "pie"  will  have  nine  (09)  slices  as  there  are  nine  per- 
formance factors.  Please  divide  the  "pie"  in  accordance  with 
the  above  example. 

FACTORS 


Speed 

Acceleration 
Wing  Loading 
Combat  Radius 
Number  of  Gun  Barrels 


Missile  Speed 

Missiles  Angle  Off  Capability 
Missile  Range 
Pilot  Experience 


As  you  divide  the  "pie"  please  number  each  slice  and  place  the 
corresponding  number  in  the  appropriate  blank. 
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SECTION  IV 


Composing  a questionnaire  is  an  inherently  difficult  task 
as  this  device  only  allows  one-way  communication.  This  problem 
inevitably  leaves  the  investigator  with  a nagging  doubt  concern- 
ing the  participants'  interpretations  of  the  posed  questions. 
Therefore,  I would  like  to  conclude  this  survey  with  a rather 
unscientific  procedure  to  determine  the  extent  of  your  inter- 
pretation difficulties,  if  any.  On  a scale  of  1 - 100,  please 
rate  the  first  three  sections  on  the  confidence  you  have  in 
your  understanding  of  the  question  (100  is  the  best) . 


SECTION  I 
SECTION  II 
SECTION  III 


I thank  you  for  your  cooperation  and  participation  in 
this  survey. 


Sincerely, 

(pJhuJl_-TY{  O'Cev \nuSL 

PATRICK  M.  O'CONNELL 

Lieutenant 

U.  S.  Navy 
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2.0  = no  response  for  upper  limit 

3.0  = no  response  for  50th  utile 
Combat  Radius 

9997.0  = no  response  for  lower  limit 

9998.0  = no  response  for  upper  limit 

9999.0  = no  response  for  50th  utile 


Note:  There  are  a total  of  200  respondants.  Each  one  has  a 

number  (1,2,3,  ...,  200)  that  is  consistent  throughout 
the  data  presentation.  Respondant  1 for  dash  speed 
is  also  Respondant  1 for  acceleration,  etc. 
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FACTOR:  NUMbER  OF  GUN  BAP  R FIS 
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FACTOR:  MISSILE  SPEED  <*ACH  NO.) 
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